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Abstract 


Entering into a new decade with an even more diversified college student population will not only require 
more assessment models involving students but also deeper professional development of institutional 
representatives key to student learning. Reflecting upon the conversations over the last three years around 
culturally responsive assessment and related equity and assessment discussions, this occasional paper 
highlights questions, insights, and future directions for the decade ahead by exploring what equitable 
assessment is and is not; the challenges and barriers to equitable assessment work; where the decade ahead 
may lead; and next steps in the conversation on equity and assessment. 
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A New Decade for Assessment: 
Embedding Equity into 
Assessment Praxis 


Erick Montenegro & Natasha A. Jankowski 


In addition to being the dawn of a new decade, January 2020 marks the three-year 
anniversary of the National Institute for Learning Outcomes Assessment’s (NILOA) 
equity conversation. Working to bring the field of assessment in alignment with 
practices that support the success of diverse learners, NILOA’s equity work focuses on 
being collaborative. The conversation on equity in assessment began with the launch of 
Occasional Paper 29 titled Equity and Assessment: Moving Towards Culturally Responsive 
Assessment. The goal of the paper was to open a dialogue on the relationship between 
equity and assessment by presenting the concept of culturally responsive assessment. 
Equity and Assessment implored the field of assessment to examine assessment processes in 
order to be responsive to both issues of equity and the needs of diverse learners, focusing 
on embedding culturally responsive assessment into processes and practices. Using 
the paper as a space for dialogue on the assumptions from which assessment operates, 
Montenegro & Jankowski (2017) sought to encourage assessment work to be reflective of 
the students served and to ultimately use assessment data to address learning, persistence, 
and attainment gaps. 


Since 2017, the message of Equity and Assessment has spread beyond the confines of 
those initial pages. While our original focus was upon exploring culturally responsive 
assessment, what we did not expect was the interest in intersections of assessment and 
equity related work. Through inviting responses to the paper introducing the concept of 
culturally responsive assessment, practitioner authors brought multiple related elements 
into the conversation, widening the dialogue space to explore the relationships between 
equity and assessment. These practice and thought leaders introduced various elements to 
consider—challenges to overcome, promising practices to move the needle, and supports 
needed in advancing equity-minded assessment work. As became clear through the 
conversations, an assessment process that is not mindful of equity can risk becoming a 
tool that promotes inequities, whether intentional or otherwise, leading to a broadening 
of the conversation from culturally responsive assessment to how assessment could address 


equity in education (Zamani-Gallaher, 2017). 


In over 15 published responses to Eguity and Assessment from May 2017 to November 
2019, respondents in the field set forth common themes in their exploration of equitable 
assessment, specifically culturally responsive assessment. For one, respondents noted 
the need for models or frameworks to inform this work and raise additional awareness 
(Henning & Lundquist, 2018a; Laird & BrckaLorenz, 2017; Tullier, 2018; McArthur 
2017; Rudnick, 2019). Questions regarding where to start, how to scale, and who 
to involve arose (Fisler, 2017), with people doing the work offering answers to these 
questions, such as beginning by disaggregating data to identify areas of need (Wright, 
2017; Williams, R., 2018), and aligning equitable practices to larger institutional goals to 
promote sustainability, alongside identifying potential partners (Levy & Heiser, 2018). 
Additionally, respondents noted considerations for culturally responsive assessment 
throughout assessment processes including the learning outcomes phase (McArthur, 
2017; Henning & Lundquist, 2018a; Levy & Heiser, 2018; Laird & BrckaLorenz, 2017; 
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Williams, E., 2018; Tullier, 2018); questions to ask during data collection to choose 
appropriate sources of evidence and include student voices in the assessment process 
(Wright, 2017; Williams & Perrone, 2018; Meyerhoff, 2019; Clark & Arimoto, 2018); 
and data analyses to check biases, dive deeper into the data, and make meaningful 
comparisons (Levy & Heiser, 2018; Williams & Perrone, 2018; Williams, R., 2018; 
Roberts, R., 2019). However, NILOA has not been the only vehicle driving the equity 
conversation forward. 


Indeed, we have seen the field respond to the call for equitable and culturally responsive 
assessment through various means. For one, we have seen an increased focus on equity 
at many higher education assessment conferences either through conference themes on 
equity, diversity, and inclusion such as the 2019 Association for Assessment of Learning 
in Higher Education (AALHE) conference and the 2020 Higher Education Assessment 
Conference sponsored by New England College, and/or specific presentation tracks 
discussing equity in assessment (e.g., 2019 Assessment Institute, 2019 Assessment in 
Higher Education Conference, and the pre-conference track at the 2020 Association 
for Institutional Research (AIR) Forum to name a few). Coupled with increases in 
publications on the topic, including special issues of academic journals devoted to equity 
in assessment (e.g., New Directions for Institutional Research Spring 2018 issue, AALHE’s 
Fall 2019 and Spring 2020 issues of Intersection). Additionally, the Council for the 
Advancement of Standards in Higher Education (CAS) and Campus Labs launched their 
Socially Just Assessment podcast, while the Student Affairs Assessment Leaders (SAAL) 
and CAS launched the Assessment for Social Justice Project (ASJP) (Henning, 2018) 
bringing together multiple organizations across higher education to explore social justice 
and assessment. We have even been fortunate to learn that various consortia of institutions 
across the country have used the concepts presented in Equity and Assessment to structure 
their assessment plans, including a group of Tribal Colleges and Universities utilizing 
elements of culturally responsive assessment to advance work within their unique mission 
and contexts. Relatedly, our equity work helped launch the Historically Black Colleges 
and Universities (HBCU) Collaboration for Excellence in Educational Quality Assurance 
(HBCU-CEEQA). Since the Fall of 2017, HBCU-CEEQA has grown to include over 
70 members from over 35 colleges and universities across the United States, focused on 
assessing student learning and being accountable to stakeholders, while also remaining 
true to the special mission of HBCUs and the students they serve (Orr, 2018). And in 
partnership with CAS and Campus Labs, NILOA released a call for equity related case 
studies to better explore culturally responsive assessment in practice, cases which will be 
released throughout the year. 


Through these important voices in the equity in assessment conversation we have learned 
much and have heard the need for further direction regarding what exactly equity in 
assessment is and is not, along with what it looks like for different stakeholders across all 
levels of higher education. It is for these reasons that as opposed to a paper focused on 


deepening and developing culturally responsive frameworks in assessment, we instead 
provide an overview of the various conversations on equity and assessment that emerged 
over the course of the past three years, with implications for future directions on where 
the larger conversation on the relationship between equity and assessment might lead in 
the decade ahead. We present what we gleaned from the insightful questions, comments, 
and perspectives shared through conference presentations, webinars, authored guest 
responses, and other literature to address: 1) what equitable assessment is and is not; 2) 
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the challenges and barriers to equitable assessment work; 3) where the decade ahead may 


lead; and 4) next steps for NILOA and the field of assessment as it relates to issues of 


equity. 
What Equity-Minded Assessment Is and Is Not 


What has become evident as more and more assessment practitioners and thought leaders 
engage with the equity conversation is the desire for a unifying definition of what is 
meant by equitable assessment. For example, NILOA’s conceptualization of equity in 
assessment, along with others (Singer-Freeman, Hobbs, & Robinson, 2019) revolves 
around culturally responsive assessment. Furthermore, CAS and Campus Labs approach 
the discussion from a socially just assessment perspective (Henning, 2018; Henning 
& Lundquist, 2018b). Thought leaders have also conceptualized equitable assessment 
through a critical perspective (Heiser, Prince, & Levy, 2017; Hanson 2019). Other efforts 
include decolonized assessment (Eizadirad, 2019), bias free assessment (Gibbs & Stobart, 
2009), assessment of learning outcomes relevant to indigenous peoples and their cultures 
(Small & Willson, 2018), and assessment that ultimately aims to do no harm. But what 
exactly does all of this mean? Questions have been posed aiming to uncover differences 
and similarities among approaches, and practitioners have requested direct translations 
of what exactly these terms mean for everyday practice. For the sake of cataloging the 
conversation thus far, brief explorations of culturally responsive, socially just, and critical 
assessment are presented below. 


Culturally Responsive Assessment 


NILOA began with cultural responsiveness for various epistemological and practical 
reasons. First, cultural responsiveness is a concept that has been associated with evaluation 
and assessment since the 1970's (Stake, 1975) and gaining prominence in the 1990's (Hood 
& Hopson, 2008). However, the conversation emerging from the evaluation community 
examined assessment and cultural responsiveness from the perspective of item validity in 
test development. Nevertheless, it provides a space from which to build upon an existing 
conversation within assessment on issues of cultural appropriateness. Additionally, the term 
“culturally responsive” is practical in nature and connected with teaching and learning. 
Stemming from Ladson-Billings’ (1995a; 1995b) seminal conceptualization of culturally 
relevant pedagogy, cultural responsiveness calls for practices which respond to the needs 
of the contexts in which we teach and learn; including the needs of the students we serve. 
If assessment is an integral and connected part of the teaching and learning process, 
then conversations on culturally responsive pedagogy are appropriately positioned to help 
better explore culturally responsive assessment and is also well aligned with NILOA’s 
prior work in transparency of assignments and assignment design (Hutchings, Jankowski, 


& Baker, 2018). 


However, the focus upon culture left readers curious. Whose culture are we being responsive 
to: the department/institution or the students? How does this translate to the tools used 
and the policies in place? Both are very important questions, but the answers depend on 
context. The reality is that culturally responsive assessment is fully dependent on the context in 
which you are assessing. It is a process that requires reflection and planning. What worked 
at one institution, program, or classroom may not work the same at another. However, 
as posited by Montenegro and Jankowski (2017), it is impossible to do without direct 
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involvement of students, noting that in its core, culturally responsive assessment must: 


1. Be mindful of the student population(s) being served and involve 
students in the process of assessing learning; 

2. Use appropriate student-focused and cultural language in learning 
outcomes statements to ensure students understand what is expected 
of them; 

3. Develop and/or use assessment tools and multiple sources of evidence 
that are culturally responsive to current students; and 

4, Intentional improvement of student learning through disaggregated 
data-driven change that examines structures, demonstrations of 
learning, and supports which may privilege some students’ learning 
while marginalizing others. 


‘These considerations guide and inform assessment work at each step of the culturally 
responsive assessment process to ensure responsiveness to the needs of students and 
implementation of meaningful improvements. Some of the elements of culturally 
responsive assessment are expanded upon in socially just assessment. 


Socially Just Assessment 


Socially just assessment stems from the work of SAAL, CAS, Campus Labs, and other 
partners in the Assessment for Social Justice Project. Socially just assessment includes the 
elements mentioned in the previous section and refocuses them within a framework that 
analyzes the interplay between culture, bias, power, and oppression in the assessment 
process. Socially just assessment calls for the acknowledgement that assessment takes 
place within various departmental and institutional cultures which impact the processes 
we follow (Heiser, Henning, & Lundquist, 2018; Henning & Lundquist, 2018b). There 
typically are norms, resource constraints, timelines, procedures in place which influence 
assessment plans and how those plans are subsequently executed. In addition, personal 
biases can influence the types of tools used, the sources of evidence to which more weight 
is assigned, and the interpretations drawn from assessment data along with possible 
solutions on how to go about improving student learning. Heiser et al. (2018) also note 
that the paradigms used to approach assessment—whether conscious or subconscious— 
work to affect decisions made and questions asked (e.g., asset-based versus deficit-based 
perspectives toward different initiatives, student populations, sources of data). 


Socially just assessment uses the concept of deconstructed assessment to not only 
understand why our students are achieving, persisting, or stopping-out in the ways 
they are, but to also understand the underpinning structures of why these things are 
happening in the first place (Henning & Lundquist, 2018b). In order to do this, there 
must be an understanding that learning and assessment operate under dynamics of power 
and oppression (Henning & Lundquist, 2018a; Heiser et al., 2018). In other words, 
assessment is not an apolitical process. We need to first understand how systems of power 
and oppression influence how students experience college, engage with the learning 
process, and build knowledge before we can understand how to better assess their learning. 
This also helps draw appropriate interpretations and conclusions from the data. Power and 
oppression can play into the assessment process when selecting whose voices to include 
in assessment and the methods we use or processes we follow. Typically, assessment is 


National Institute for Learning Outcomes Assessment 


planned and carried out by faculty and administrators, and changes are implemented 
according to what faculty and administrators assume to be most appropriate. Seldomly 
are students involved to verify that the assumptions in play are real, appropriate, or meet 
their needs. 


The perspectives we include in assessment and decisions of who has a voice at the 
table privileges certain ways of knowing while potentially oppressing those who are 
not represented. For example, culturally responsive assessment stresses that the student 
voice must be included in assessment because students have been typically treated as 
the object of the assessment; a mere passive participant in a process that has important 
ramifications for their success (Montenegro & Jankowski, 2017). As argued by Zerquera, 
Reyes, Pender, and Abbady (2018), “Many of the assessment approaches employed today 
are misaligned with social justice agendas, failing to adequately inform decisions about 
how best to support marginalized student populations within higher education” (p. 17), 
reinforcing the need to actively involve students throughout the process of assessment. 
With this in mind, socially just assessment raises awareness on how assessment can be a 
process inherent of structures of power and oppression (Henning & Lundquist, 2018b). 


Socially just assessment reminds practitioners to be mindful of how the ways in which data 
are analyzed can also privilege or oppress. Far too often, if a specific student population 
has a small sample size in assessment data, they are removed from analyses (American 
Indian College Fund, 2019). This inherently marginalizes specific populations because 
they are silenced from analyses, even though much can be learned from the experiences 
of the students behind the small ‘n’ (Montenegro & Jankowski, 2017; Heiser et al, 2018). 


Furthermore, assessment needs to be more aware of the types of comparisons made 
between learners (Levy & Heiser, 2018). Far too often, we compare the outcomes of 
students of color to those of white students. White students are then normed as the 
population to which others should strive. Or we examine theories of white male student 
persistence, pipeline approaches to education and learning, and assume them to be 
the theory that should guide our practice to help “the other students” be more like the 
“successful students.” These comparisons, especially if not worded or contextualized 
appropriately, can send the message that non-white students should strive to be like their 
white peers without examining the unique experience of non-white students. The point 
raised in social justice focused assessment is that “cultural and social differences influence 
whether and how students perform academically and socially at their institution” (Dorimé- 
Williams, 2018, p. 42) as well as that “flawed assessment and implementation processes 
disadvantage students” while “inaccurate interpretation and reporting of results can lead 
to policies with a discriminatory impact” (Dorimé-Williams, 2018, p. 51). 


The end goal of socially just assessment is to advance social justice. In other words, 
assessment should strive to serve as a mechanism that helps close opportunity, persistence, 
and attainment gaps between different student populations. Socially just assessment 
should challenge structures of privilege within institutions and society writ large to better 
serve and support learners. The goal of assessment is to make data informed decisions 
on how to improve teaching and learning, so the goals of social justice and assessment 
are very similar. It takes a conscious, intentional approach to make it happen, alongside 
potentially hard conversations. Indeed, socially just assessment echoes the same values 
expressed in culturally responsive assessment but bounds them within an exploration of 
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how power and oppression impact assessment. 
Critical Assessment 


At its most fundamental level, equitable assessment requires approaching assessment 
through a critical lens. Both culturally responsive assessment and socially just assessment 
operate from a critical perspective. They strive to challenge the status quo; raise questions 
of privilege, power, and oppression; and work to remedy injustices whether purposeful or 
accidental. Heiser et al. (2017), as well as Hanson (2019) brought forth various elements 
of critical inquiry which can be applied to the assessment process. In short, critical 
assessment calls for: 


1. Disregarding the objectivity myth and accepting that assessment is 
inherently subjective and guided by the biases and experiences of those 
conducting assessment; 

2. Varying the types of evidence used to assess learning outcomes to not 
privilege specific ways of knowing or preferred ways to demonstrate 
knowledge; 

3. Including the voices of students, especially those who belong to 
minoritized populations or those whose voices can often be left unheard, 
throughout the assessment process; and 

4, Using assessment to advance the pursuit of equity across previously 
identified institutional parameters that demonstrate disparate outcomes 
across student populations. 


What these four tenets outline is the important role that context plays in critical 
assessment. The context of the institution/program, the person(s) conducting the 
assessment, and the learners need to be understood and reflected upon in order to 
properly create and execute an assessment plan that will yield appropriate and equitable 
results. However, simply being attentive to issues of equity during the assessment process 
is not enough. Results must then be used to improve equity imperatives for the student 
populations experiencing inequitable outcomes, in part because the learning outcomes 
that institutions list are learning outcomes to which all students strive. Thus, assessment 
efforts must be consequential to issues of equity. Culturally responsive assessment and 
socially just assessment centralize these tenets of critical assessment. There certainly are 
nuances between them, but they each strive to meet the same end goal: increase equity 
in assessment. 


Bringing it All Together: Equity-minded Assessment 


Nomenclature aside, each of the above perspectives on equity and assessment are rooted 
in the same core notion of being mindful of equity and actively working to address 
inequities. At its core, equitable assessment calls for those who lead and participate in 
assessment activities to pay attention and be conscious of how assessment can either 
feed into cycles that perpetuate inequities or can serve to bring more equity into higher 
education. From this point on, this paper will use the term equity-minded assessment, 
similar to how Bensimon (2006) speaks of equity-mindedness, as it encompasses a shared 
perspective on equity across the various conversation spaces unfolding. Here, we draw out 
principles and elements of culturally responsive assessment, socially just assessment, and 
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critical assessment that have emerged over the past three years of conversation. 
1. Meaningful Student Involvement 


Being mindful of equity brings a necessary and embedded focus to questioning the 
assessment process to ensure we are not succumbing to biases or established norms 
while simultaneously excluding important voices and perspectives. One of the easiest 


means by which to check assumptions is to actively involve students in the process of TTT 


assessment. In the face of changing enrollments in higher education, continued and 
widening opportunity, persistence, and attainment gaps for different student populations 
(Condron, Tope, Stiedl, & Freeman, 2013), the “typical” or “traditional” way is simply not 
working for specific groups of students. Equity-minded assessment is about challenging 
what we think and exploring what others think, need, and are affected by which we may 
not understand or experience. Just as authentic assessment requires the use of multiple 
sources of evidence (Kuh et al., 2015), equity-minded assessment invites multiple voices 
to the table to determine learning outcome statements (Tkatchoy, 2019), inform the 
appropriateness of assessment (Gipps & Stobart, 2009), and make sense of the results. 


A good starting place is with statements of learning outcomes—both in ensuring they are 
measurable and that faculty, staff, and students understand and can make sense of those 
outcomes. If learning outcome statements serve as the point from which educational 
experiences are designed, and the learning outcome statements themselves are not inclusive 
or include biases, then the educational design will as well (Rodrigues & Raby, 2019). 
Further, students can be involved in determining what could be changed to further their 
learning. Institutions are positioned to make more impactful changes by engaging with 
students about what would best support their learning as opposed to trial and error, or 
implementing “what worked for me when I was a student.” Experiences and perspectives 
come with biases which can be embedded into the assessment process. If we do not reflect 
on these biases, and take action to challenge them, then we risk acting upon assumptions 
which may not be appropriate for students—even with the best intentions. 


Listening to the voices of those historically silenced is an essential element of equity- 
minded assessment. Fully complementing this is ensuring that everything, from learning 
outcomes to data collected and reports are 1) written in a way that can be understood 
by students and other stakeholders; and 2) are disseminated and communicated through 
channels which can be easily found by students and stakeholders. In other words, equitable 
assessment is transparent. 


A forthcoming equity case study of Capella University, notes that equity-minded 
assessment requires transparency in the assessment process as well as in educational 
design—students should know what is being assessed, how it is being assessed, and how 
well they achieved the assessment’s expectations. If students are to be active participants 
in assessment, then we need to ensure students are also informed of assessment results, 
improvements or changes made due to assessment data, and what this means for students. 
Capella believes, equitable assessment practice means that all learners have an equal 
and unbiased opportunity to demonstrate their knowledge and achievements through 
assessment processes that use differentiated methods, transparency, accountability, and 
fairness from design to measurement to improvements and dissemination of results. 
(NILOA’s Transparency Framework can offer additional insight on how to make 
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assessment a more transparent process by making information accessible and engaging.) 
2. Data Disaggregation, Exploration, and Action 


A second element of equity-minded assessment involves exploring assessment data to 
uncover potential learning gaps between student populations to make data informed 
changes in order to close those gaps. In other words, equity-minded assessment requires 
meaningful data disaggregation and subsequent action. In reviewing data, students can be 
an active part of the conversation such that their needs and lived realities are present, heard, 
and acted upon. Consider the noise added to data on a particular learning outcome if 
collected results are not a demonstration of students’ learning on that particular outcome 
but are instead based on social capital related to navigating assessment tasks? Should 
curricular changes be made based on data about structural inequities as opposed to 
learning? Another forthcoming equity case study from the University of North Carolina 
at Charlotte notes ways to identify false evidence of achievement gaps in data analysis. The 
exploration revealed that differences in student grades were reflective of students’ ability to 
navigate college instead of demonstration of competence or achievement of an outcome. 
This led assessment professionals to turn their attention to making assignments more 
equitable for assessing learning instead of the “hidden curriculum’; a similar sentiment 
echoed by R. Roberts (2019) who urged assessment practitioners not to make decisions 
from assessment data which only reflects students’ ability to navigate assessments (e.g., 
ask questions and have good test-taking skills) instead of their actual learning. 


As learned from the National Association of System Heads (NASH) project on taking 
high-impact practices to scale, meaningful data disaggregation is a good first step towards 
examining equity issues but is not by itself a practice of equity-minded assessment. 
Simply examining disaggregated data without examining if the assessment process is 
equitable will lead to continued inequities. Meaningful disaggregation involves deeper 
analyses by specific student characteristics, alongside the intersection between and among 
them (Roberts, J., 2019). To enable a place where practitioners can dive deeper into the 
data, we first need to gather assessment data at a level and in a manner which can be 
meaningfully disaggregated. It is incredibly difficult to disaggregate data at an institution- 
level if data do not exist. To address issues of disaggregation, institutions might work in a 
data sharing consortium in order to explore disaggregation options, or instead undertake 
focus groups with students by verbally exploring differences in experience. 


‘The first step is evaluating the depth of assessment data on hand and what can realistically 
be done with it. Before collecting more data, it is always wise to fully explore the data 
already collected and determine if it should still be collected or if something else is needed. 
This can be done in partnership with institutional research or institutional effectiveness 
staff as well as assessment professionals (should you be at an institution with so many 
positions!). Further, we should consider siloed data which may provide additional insight 
if connected to data on student learning whether in the Student Information System, 
Learning Management System, course related systems, or other data sets managed by 
student affairs and related units. Then, we can begin to determine what analyses to run. 


An assessment plan that can yield the data needed would include shared definitions, 
variables, and student characteristic data, which takes time and conversations to 
determine. We cannot expect to collect all of the data in one round of assessment, but 
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over time we can begin to collect data that allows us to explore the intersection of various 
student characteristics. For example, a deeper dive into our data can show that low- 
SES male Latinx students are disproportionately less likely to attain a specific learning 
outcome compared to both their low-SES female Latinx peers and their high-SES Latinx 
counterparts. Or we might find that low-SES full-time commuter students are lagging 
behind both their full-time residential peers and their high-SES full-time commuter peers 
in attainment of a particular learning outcome in certain programs based on curricular 
paths. Here, both SES and the residential/commuter characteristics are of importance, 
coupled with student voice as to why this might be the case. 


A cautionary note: In equity-minded assessment, data related conversations should be 
interrogated to ensure that data are not weaponized to facilitate self-fulfilling prophecies 
where results are used as “proof” that students are unprepared or disinterested; where 
someone can take assessment data and say “see, I told you they can’t learn!” To support 
equity-minded assessment discussions informed by data, professional development may 
be needed prior to data discussions to interrogate biases and assumptions. An important 
aspect of meaningful disaggregation is thus knowing which questions to ask and what 
to do with the different findings, as well as who to have participate in the process. This 
takes practice, patience, and thrives from collaborating with and learning from others 
who have different experiences, skills, and perspectives than our own. For faculty and 
assessment practitioners to become comfortable with such conversations, administrators 
need to provide spaces with facilitated discussions on structural barriers, inequities, and 
practices which can affect student outcomes. Gansemer-Topf, Wilson, and Kirk (2019) 
offer various questions assessment practitioners can ask of the data to critically interrogate 
data collection and analysis processes. 


Second cautionary note: While small samples can inform assessment through trend 
data collected over time, equity-minded assessment is responsive to student needs by 
examining and helping individual students, when they need it (Maki, 2017a). We cannot 
disregard data simply because it is from a small sample. Generalizability is important for 
publications but not as much for equity-minded institutional improvement or decision- 
making. If the capstone for a major/program has three students in it who will graduate 
and all three students struggled with attaining the program learning outcomes, should 
something be done, or is the “sample size” (in this case the population of graduates — not 
a sample, but an often confused point in these conversations) too small to incite action, 
when none of the graduating students met the learning of interest? If ten students out of 
500 are not meeting an outcome, it may not be statistically significant, but it is significant 
to those students’ and their families and has implications for retention and persistence. 
Thus, it is worth exploring. Further, looking at the same student in multiple points 
through their learning journey in relation to learning outcomes provides information 
not simply on one data point at the end of a program, but many throughout, in order to 
examine learning progression over time. Such an approach shifts the unit of analysis from 
a student to instances of demonstrated learning, thus one ‘n’ may have many associated 
data points. 


3. Context-Specific Approaches and Responses 


Third, equity-minded assessment requires that we address issues of equity within our 
specific context. It is impractical and unlikely that assessment professionals working to 
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advance equity and assessment within their institutions will be able to fix the entirety of 
the educational system in the United States. In addition, there are many areas and issues 
to explore within a specific institutional context on equity and assessment, and it is unwise 
to tackle them all at once. Determining an issue of equitable assessment that can bring 
faculty, staff, and students together in a space of productive discomfort will lead to more 
equitable assessment in the future than charging ahead with full disruption of assessment 
processes in the name of equity. Some possible ways to begin include exploratory 
analyses—done to see if there are inequities within assessment data and inquire about 
how they can be fixed—or purposeful analyses to see if a recent change intended to 
close a learning outcomes gap among specific populations achieved its intended purpose. 
Starting with changing student demographics as a point of exploration of learning success 
or of questions that faculty have about students and their learning opens space to examine 
learning while bringing an intentional equity lens to the discussion. Additionally, equity- 
minded assessment efforts can align to ongoing department, program, or institutional 
initiatives to help meet overarching student success and learning goals, thus helping 
to inform a larger issue that the institution needs to address or is already addressing. 
Whatever the case, assessment efforts must be mindful of inequities which matter in a 
specific context and assessment professionals need to sensitively navigate institutional 
initiative space, working to make the case and connect the dots between equity and 
assessment for faculty, staff, and administrators (Jankowski & Slotnick, 2015). 


4. Embedded in All Things Assessment 


Finally, equity needs to be embedded within and throughout the entirety of any assessment 
effort. In her Assessment Institute keynote remarks, Tia B. McNair (2019) said that those 
doing equity work need to /ive equity work. In other words, doing equity work is not 
something we can step in and out of. It is a mentality and approach that remains central 
so that we do not lose sight of it, that others are able to follow by example, and we are 
always being critical, reflective, and questioning processes, biases, assumptions, within 
ourselves, others, and the processes followed. This equity-mindedness needs to actively 
permeate the entire assessment process, and the practice of assessment professionals. To 
do equity-minded assessment we need to: 


1. check biases and ask reflective questions throughout the assessment 
process to address assumptions and positions of privilege; 

2. use multiple sources of evidence appropriate for the students being 

assessed and assessment effort; 

include student perspectives and take action based on perspectives; 

increase transparency in assessment results and actions; 

5. ensure collected data can be meaningfully disaggregated and 
interrogated; and 

6. make evidence-based changes that address issues of equity that are 
context-specific. 


i 


Equity-minded assessment refers to ways we ensure assessment processes and practices are 
appropriate for a// students and that we ultimately do no harm in the process. While it 
can be challenging to consider the vast differences and needs of our student populations 
in our practices, our task as educational providers is to strive to help every student succeed. 
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What “ensuring that our assessments are appropriate for a// students” means is that 
equitable assessment should work to ensure that learning outcomes, and how we assess 
those outcomes, are done in ways which do not privilege certain students over others; 
that data-informed changes are not benefiting one student group over others; and that 
assessment efforts are not conducted with only one dominant perspective or voice leading 
the process. Once we are aware of inequity in learning or assessment, we should strive 
to address it instead of ignoring it; or worse, blaming students. Equitable assessment 
means that we interrogate changes for possible disparate impacts on different student 
populations and their learning, that we examine the changes with data collected on various 
characteristics of interest to examine if learning improved, and provide students with 
multiple opportunities to advance in their learning before leaving our institutions (Maki, 
2017a). This means that the learning gains we desire are not only for future students, but 
the students who are in the active process of learning (Maki, 2017b). 


The Barriers and Challenges to Equity in Assessment 


Assessment with an embedded focus on equity is attainable, but there are barriers to 
advancing the work. Some of those barriers deal with discomfort with engaging in 
conversations about privilege, power, oppression, and marginalization. In her book White 
Fragility: Why Its so Hard for White People to Talk About Racism, DiAngelo (2018) explores 
the various cognitive barriers some can encounter when equity and related concepts are 
discussed. With the many lessons of this book in mind, R. Roberts (2019) notes that 
assessment practitioners who avoid equity simply because it is uncomfortable or an 
inconvenience makes assessment practitioners accomplices to practices which perpetuate 
inequities; especially through sustaining “barriers to student success, retention, graduation, 
and, most of all, learning” (p. 3). In her response to Equity and Assessment, R. Roberts 
(2019) provides a rich personal experience that exemplifies this behavior: 


I recently attended a training focused on equity and data-informed 
improvements to instruction... The group excitedly followed along during the 
first day of our training when we reviewed information about the efficacy of 
the proposed improvements that we could all bring to our colleges. However, 
when the presentation shifted to a review of national data about inequity, the 
room exploded with anxiety. Suddenly, several white people had seemingly 
random objections, others had comments they believed were crucial about 
all the aspects of inequity outside of their control, or comments about how 
“other faculty” on their campuses would never tolerate reflecting on campus- 
wide or course-specific data on inequity. The presenter repeatedly had to 
intervene to redirect the conversation... She reminded us of the challenging 
but extremely important truth that there are some things as instructors and even 
administrators that are absolutely within our control when it comes to improving 


equity (emphasis added) (pp. 1-2). 


The last sentence is invaluable. Equity work is not someone else’s responsibility. We each 
play a role in equity-minded assessment. There are elements we control within our spheres 
of work and influence which can alleviate the mechanisms through which inequities exist 
and persist. However, responsibility is diverted with comments such as “this is not a 
problem at my institution” or “I can see why this would matter for Minority-Serving 
Institutions but not us.” Choosing not to see an issue does not mean it is not still there, 


Equitable assessment 
should work to ensure 
that learning outcomes, 
and how we assess those 
outcomes, are done in 
ways which do not privi- 
lege certain students over 
others. 
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nor can only one type of institution approach assessment in an equity-minded perspective. 


A very real barrier to this work, which most can relate to, is initiative fatigue (Kuh & 
Hutchings, 2015). At many institutions, people are wearing multiple hats often being the 
lifeline of various projects, students, and other stakeholders in addition to their daily job 
responsibilities. For those who work in assessment offices, most are often understaffed, 
without much authority to convene faculty or directly impact policy and practices across 
the institution. Assessment professionals have to contend with accreditation requirements 
and various external calls for accountability while addressing issues of ongoing need 
internal to the institution. Being asked to be the “assessment person” supporting one more 
institutional initiative can be a significant load to bear. Adding further responsibility to 
be the sole driver of equitable assessment can further stretch thin assessment staff, thus, 
finding partners in the work before implementation can be key to further success. 


Institutional culture plays an important role too; specifically the culture around assessment. 
For example, if the assessment culture is about compliance/reporting or positivist-based 
scientific views of measurement, then equity might not be an important part of the 
conversation aside from item validity or reporting required disaggregated data (Jankowski, 
2017). However, if the culture of assessment is focused around improving learning and 
teaching, then equity can be a fruitful approach. The point being that depending on 
the culture of assessment, the conversation around equity-minded assessment can be 
difficult; often requiring someone to lead and facilitate the discussion in language that 
resonates with the institutional context. An example of such facilitated approaches are 
Assignment Charrettes— intensive assignment design workshops that are led by faculty— 
or the Transparency in Learning and Teaching in Higher Education (TILT Higher Ed) 
project that helps faculty add transparency in their pedagogy to improve student learning 
(Winkelmes, 2016). This way, our assessments and the use of assessment data can both 
lead to more equitable learning outcomes. 


Another challenge is determining how much to involve students in the assessment process, 
coupled with the comfort-level of faculty, staff, and administrators regarding student 
involvement in assessment. Assessment practitioners, in partnership with others within 
the institution, will need to determine how much and in what ways to involve and recruit 
students to be part of different stages of the assessment process. While students may 
initially lack assessment literacy (Smith, Worsfold, Davies, Fisher, & McPhail, 2013), or 
an understanding of assessment, student involvement in assessment as active participants 
with agency over their learning has proven beneficial to student learning and the overall 
student experience (Jankowski, Baker, Brown-Tess, & Montenegro, forthcoming; Singer- 
Freeman & Bastone, 2019). While recruitment of students may be difficult—because 
just like faculty and staff students wear multiple hats and are rather busy— students may 
be more likely to participate if they understand the impact(s) and/or benefits of their 
involvement. No one wants to waste their time or, worse, share their thoughts and see no 
action taken in response; thus adding to feelings of being unheard or unseen. 


As the roles of students in assessment are explored for a particular institution, measurement 
related concerns towards active involvement of students in the design, administration, 
and analysis of assessment related information may arise. Most of these concerns stem 
from conflicts of interest and issues of objectivity and bias from involving the students 

who are being assessed in the process of measuring their own learning. However, student 
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involvement in assessment at an institution-level involves a variety of students in the 
process (Maki, 2017b; Damiano, 2018), and involving students as part of the assessment 
process within a course can position assessment as a formative opportunity for learning, 
not just summative demonstrations (Hattie, 2009). Involving students in the process 
of measuring learning supports learning outcomes of quantitative reasoning along with 
written and oral communication as well as involves students in undergraduate research 
(Truncale et al., 2018; Welsh, 2013). However, equity will not be attained through 
placement of a token student on institutional assessment committees or connections 
with student governance without a wider involvement of students from throughout the 
institution. 


Looking to the Decade Ahead 


We are not tasked with changing the world with one assessment effort or verifying that 
learning outcome statements are culturally appropriate with every individual student. We 
are not expected to survey every single student about intended data-informed changes, 
nor does equity-minded assessment call for every single student to participate in data 
analysis. Instead, equity-minded assessment is ultimately about being responsive, aware, 
and intentional in order to not perpetuate inequalities. 


We are aware of the various challenges to this work. In part, there are data issues including a 
lack of common definitions by which institutions gather and report student characteristics, 
coupled with a lack of data by which to disaggregate—in ways that intersect with multiple 
student identities— but also intersect with data on learning. In part, there is a design issue 
for embedding equity in assessment as well as the means by which to be transparent to 
students about the assessment and learning process. While some faculty and staff actively 
engage in universal design for learning and structure curricular and co-curricular learning 
experiences such that access and success are not student characteristic dependent, it is not 
widespread enough to occur for every student, every time. And finally, in part we are not 
measuring in ways that provide the data needed to address issues of equity. It is to address 
these challenges in the coming decade that we turn. 


Professional Development: The Key to Unlocking the Potential of Equity-Minded Assessment 


Professional development can break down barriers to equity-minded assessment. It 
can bridge perceived gaps between assessment and context (Levy & Heiser, 2018), and 
between knowledge and knower. We cannot assess what students know without also 
attempting to understand how culture, context, and the influence of both impact learning 
and how we assess that learning (Fisler, 2017; Montenegro & Jankowski, 2017). Levy and 
Heiser (2018) suggest that “Institutions may want to create a meta-assessment rubric or 
checklist to help ensure assessment practice is following proper process as intended by the 
institution in accordance with institutional goals and values” (p. 3). The goal being that by 
maintaining a clear vision on good assessment practice, biases may be limited. However, 
Tharp (2019) argues the need for individual professional development on issues of equity, 
making the case that for issues of equity to be examined within an institution, individuals 
must spend time working on understanding their own assumptions and biases first. Thus, 
we need to both examine processes and practices and ensure there are checkpoints on 
implementation, as well as provide support for individuals to explore issues of equity in 
order to meaningfully implement processes and practices in equitable ways. 


We need to both 
examine processes and 
practices and ensure 
there are checkpoints on 
implementation, as well 
as provide support for 
individuals to explore 
issues of equity in order to 
meaningfully implement 
processes and practices in 
equitable ways. 
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The truth is that many of us are not comfortable with or well-versed in conversations about 
equity. However, the best way to acquire a skill is not by avoiding it, but by immersing 
ourselves in it. Having conversations about race/racism or power/oppression—even if 
they are just reflective and introspective—can be uncomfortable. None of us want to find 
that our practices may be marginalizing students or contributing to inequitable outcomes. 
But that fear—or worse yet, complacency and comfort in current practices because they 
work for “most” students—prohibits progress and perpetuates inequities and inequality. 
Addressing equity is everyone’s job at the institution, and difficult conversations cannot 
be passed to someone else. Equity work requires assessment professionals to be courageous 
and continuously seek ways to develop skills in equitable assessment, but everyone in the 
institution has a role to play. 


Professional development is an important partner in equity work because it allows faculty, 
staff, and administrators the space to improve their own learning and understanding 
in order to improve the learning of students; all the while helping programs and 
institutions better meet student success goals. Equity focused professional development 
should be communicated as important and supported with targeted incentives to engage 
all stakeholders, not just the willing. As R. Roberts (2019) explains, “To develop this 
self-awareness...educators should participate in ongoing training in understanding and 
supporting equity and how it relates to inquiries about culturally-responsive outcomes, 
classrooms, and professional development” (p. 3). In truth, doing equity work is a 
continuous process—much like assessment—because our student populations change 
with enrollment trends, and gaps in learning change continuously. This means that 
equity work, and especially our own continuous development in this area, is a life-long 
process that requires “sustained engagement, humility, and education” (DiAngelo, 2018, 
p. 9) regardless of race/ethnicity and background. Equity-minded assessment requires a 
certain comfort with being uncomfortable; with having tough conversations, engaging in 
reflective practices, and implementing a critical mindset throughout. 


Relatedly, we could all benefit from professional development on how to involve student 
voice to ensure biases are in check and equity is embedded throughout assessment 
practice. Biased assessments fail students. They can be unfair to learners who are not fluent 
in specific cultural norms and exclude the experiences of linguistically, culturally, and 
socioeconomically diverse learners (Williams & Perrone, 2018). Professional development 
helps practitioners engage in practices that maintain a conscious understanding of how 
practices and decisions are influenced and, in turn, influence the assessment effort. 


In the coming decade, research is needed on effective equity focused professional 
development supports for various levels within an institution. What might be the 
role of centers for teaching and learning in working with faculty in partnership with 
assessment professionals on issues of equity and assessment in the classroom? Note: 
see Levesque-Bristol et al., 2019 and Kinzie, Landy, Sorcinelli, & Hutchings, 2019 for 
some ideas on how centers for teaching and learning could be involved. Where will 
assessment professionals find support on issues of equity-minded measurement and 
data collection? What is the role of institutional researchers in equitable assessment and 
what professional development supports will they need? What professional development 
might accreditation related positions need to address issues of equity? And where will 
administrators find models for supporting equitable assessment through professional 
development? It is our hope that the decade ahead provides answers and resources to the 
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wide-scale professional development needs to fully support equity-minded assessment 
across an entire institution's assessment process. 


Focusing on Equity and Assessment 


Since the launch of the equity and assessment conversation, many threads of related 
discussion have emerged. In the coming decade, assessment professionals will 
continue to unpack issues of equity and measurement; data definition, collection, 
and disaggregation; decolonizing assessment and learning outcomes; and indigenous 
approaches to assessment. Student affairs will lead in socially just assessment, and groups 
such as HBCU-CEEQA will provide insight into the equity-minded practices that have 
been unfolding for years within the confines of their member institutions but have not 
been acknowledged or implemented more broadly at predominantly white institutions. 
And while the use of multiple sources of evidence can be helpful to culturally responsive 
assessment, it should not be taken as a simple solution, and instead an opportunity to 
explore equity in design and measurement and how to offset different sources of bias, if 
possible. It is our intention that over the course of this decade, we will work to elevate 
the institutions who have been doing equitable assessment and have models and answers 
to scholarly questions, but whom have been silenced or not asked to join the assessment 
conversation thus far. 


We expect a challenging of measures of institution-level assessment around areas such 
as climate, which have been historically presented in ways that ensure “white” students 
are comfortable and experiencing “enough” diversity, as opposed to understanding 
diverse student experiences or what an equitable climate entails (Phillips & Jones, 
2019). And if our practice is guided by theory and our theories are inequitable, we 
have a responsibility in the coming decade to develop theories that address as well as 
interrogate the norms around student behavior, engagement, and what a “good” student 
does to demonstrate their learning in ways that address diverse student populations and 
their experiences. Discussions on how to embark on equitable comparison groups along 
with related supports and possible changes to see success are all rife for unpacking in 
the decade ahead—if as a discipline of assessment we focus on equity and assessment, 
at all levels of assessment from classroom to program to college to general education 
and institution. We will continue to see examples of ways in which students can be 
involved in assessment, ranging from curating their own collections of evidence related 
to learning outcomes, participating in transparent assessment design, or simply helping 
to rewrite learning outcome statements in student-focused language. It is not enough 
to tell our students about the intentional design through transparency approaches if 
the design itself remains flawed and inequitable. Assessment is an ongoing process of 
improvement helping to continuously refine teaching and learning, as well as assessment 
processes and practices, and a focus on equity can help us attain this goal. 


Final Thoughts 


‘The equity in assessment conversation is far from over and it will become an increasingly 
important practice for higher education as our student populations continue to diversify. 
NILOA will continue to embed equity-minded assessment as a central thread in our 
efforts. This is especially true as we continue our partnerships with others working in 
the equity space. We pledge to continue the dialogue on equity-minded assessment and 


We have a responsibility 
in the coming decade 

to develop theories 

that address as well as 
interrogate the norms 
around student behavior, 
engagement, and what a 
“good” student does to 
demonstrate their learning 
in ways that address diverse 
student populations and 
their experiences. 
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to continue supporting and learning from the field so that we may advance this work 
together. We must all reflect on how our privilege(s) and positionality within society, the 
institution, and the classroom intersect with that of students and assessment processes 
and practices. We look forward to continued reflection by assessment professionals on 
the ways that current assessment efforts either centralize issues of equity or serve to 
perpetuate them (Felder, 2017), showcasing examples of implementation, and pushing 
the scholarly conversation forward towards wider understanding and action. 
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